{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# In-Context Learning using OpenAI API"
      ],
      "metadata": {
        "id": "M9ce4ndDNsTr"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In-context learning, also known as few-shot learning, is a powerful technique in natural language processing that leverages pretrained foundational models to perform complex tasks without the need for explicit retraining or parameter updates. This method enables models to quickly adapt to new tasks by using a limited number of input-output examples provided directly in the model's prompt. This approach helps guide the model in understanding how to process and respond to new types of inputs effectively.\n"
      ],
      "metadata": {
        "id": "7_7fUdVUNwbZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let’s have a look at the Banking-77 Dataset which contains online banking queries annotated with their corresponding intents"
      ],
      "metadata": {
        "id": "5qjS-508N2E6"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "PGMa2WGxNfRe",
        "outputId": "1739637f-97af-4462-bfae-afc9ade2acad"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                text  \\\n",
              "0  i moved to a new city and need to change my ad...   \n",
              "1       on my transfer there was a \"decline\" message   \n",
              "2  help!  my wallet was stolen and someone is tak...   \n",
              "3  while abroad i got cash, and a wrong exchange ...   \n",
              "4                              why can't i get cash?   \n",
              "\n",
              "                                     label  \n",
              "0                    edit_personal_details  \n",
              "1                        declined_transfer  \n",
              "2                 card_payment_fee_charged  \n",
              "3  wrong_exchange_rate_for_cash_withdrawal  \n",
              "4                       getting_spare_card  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-aaab696a-7405-4ae6-b943-31a4139c1cdd\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>i moved to a new city and need to change my ad...</td>\n",
              "      <td>edit_personal_details</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>on my transfer there was a \"decline\" message</td>\n",
              "      <td>declined_transfer</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>help!  my wallet was stolen and someone is tak...</td>\n",
              "      <td>card_payment_fee_charged</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>while abroad i got cash, and a wrong exchange ...</td>\n",
              "      <td>wrong_exchange_rate_for_cash_withdrawal</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>why can't i get cash?</td>\n",
              "      <td>getting_spare_card</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-aaab696a-7405-4ae6-b943-31a4139c1cdd')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-aaab696a-7405-4ae6-b943-31a4139c1cdd button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-aaab696a-7405-4ae6-b943-31a4139c1cdd');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-0b34f4e9-276b-4c03-9b22-b37e7932fdd9\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-0b34f4e9-276b-4c03-9b22-b37e7932fdd9')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-0b34f4e9-276b-4c03-9b22-b37e7932fdd9 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"examples_pool[['text', 'label']]\",\n  \"rows\": 5,\n  \"fields\": [\n    {\n      \"column\": \"text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"on my transfer there was a \\\"decline\\\" message\",\n          \"why can't i get cash?\",\n          \"help!  my wallet was stolen and someone is taking money out.  i need this money!  what can i do?\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"label\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"declined_transfer\",\n          \"getting_spare_card\",\n          \"card_payment_fee_charged\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 3
        }
      ],
      "source": [
        "import pandas as pd\n",
        "\n",
        "test = pd.read_csv('https://s.cleanlab.ai/banking-intent-50/test.csv')\n",
        "examples_pool = pd.read_csv('https://s.cleanlab.ai/banking-intent-50/examples-pool.csv')\n",
        "\n",
        "examples_pool[['text', 'label']].head()\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Building the Few-Shot Prompt\n"
      ],
      "metadata": {
        "id": "UYKXYL9XOAjU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Prefix - List of Classes for Valid Completions\n",
        "This text will go at the beginning of the prompt and will tell the LLM what the valid classes are so that it can consistently output a class. Without this, the LLM will not choose a valid class and output something not parsable.\n",
        "\n",
        "\n",
        "Here we can also add an optional prefix that we will use later.\n"
      ],
      "metadata": {
        "id": "An3yOOUMOC1J"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Helper to get prefix for prompt. This gives the LLM all of the labels so that it chooses more accurately.\n",
        "def get_prefix(examples_pool, prefix=\"\"):\n",
        "    s = \"\"\n",
        "    if len(prefix) != 0:\n",
        "        s += prefix\n",
        "        s += '\\n'\n",
        "    s += \"You can choose the label from: \"\n",
        "    classes = list(examples_pool.label.unique())\n",
        "    s += \",\".join(classes)\n",
        "    return s\n",
        "\n",
        "print(get_prefix(examples_pool, \"Beware some labels in the examples may be noisy.\"))\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VomcbEntNgdc",
        "outputId": "d55267ff-bed2-43e3-a0f4-7d7e703a96b1"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Beware some labels in the examples may be noisy.\n",
            "You can choose the label from: edit_personal_details,declined_transfer,card_payment_fee_charged,wrong_exchange_rate_for_cash_withdrawal,getting_spare_card,cash_withdrawal_charge,verify_top_up,transfer_timing,apple_pay_or_google_pay,card_payment_not_recognised,visa_or_mastercard,reverted_card_payment,transfer_not_received_by_recipient,country_support,wrong_amount_of_cash_received,refund_not_showing_up,card_linking,failed_transfer,exchange_via_app,fiat_currency_support,activate_my_card,direct_debit_payment_not_recognised,balance_not_updated_after_cheque_or_cash_deposit,cash_withdrawal_not_recognised,transfer_fee_charged,card_arrival,pending_top_up,extra_charge_on_statement,supported_cards_and_currencies,declined_card_payment,top_up_failed,automatic_top_up,transaction_charged_twice,disposable_card_limits,card_payment_wrong_exchange_rate,pending_transfer,declined_cash_withdrawal,balance_not_updated_after_bank_transfer,beneficiary_not_allowed,why_verify_identity,order_physical_card,request_refund,card_about_to_expire,cancel_transfer,pending_card_payment,change_pin,lost_or_stolen_phone,pending_cash_withdrawal,top_up_reverted,exchange_charge\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Providing K-Shot Examples\n"
      ],
      "metadata": {
        "id": "DQr_EHonOULZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Here we randomly choose 50 examples, 1 from each class to build a 50-shot prompt for the LLM.\n"
      ],
      "metadata": {
        "id": "QhJWj6DBOZYB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Helper method to get one example from each class for k-shot prompt.\n",
        "import random\n",
        "def get_examples(examples_pool):\n",
        "    out = []\n",
        "    unique_classes = examples_pool.label.unique()\n",
        "    for i, cls in enumerate(unique_classes):\n",
        "        temp = examples_pool[examples_pool.label==cls]\n",
        "        random.seed(0+i)\n",
        "        idx = random.choice(list(range(len(temp))))\n",
        "        text = temp.iloc[idx].text\n",
        "        label = temp.iloc[idx].label\n",
        "        d = {'text':text, 'label':label}\n",
        "        out.append(d)\n",
        "    return out\n",
        "\n",
        "examples = get_examples(examples_pool)\n",
        "examples[1:5]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OfL5w9-KOP4L",
        "outputId": "e60dc3d8-f59c-419f-f0c0-c37753a9a253"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[{'text': 'it declined my transfer.', 'label': 'declined_transfer'},\n",
              " {'text': \"why am i being charged for atm cash withdrawals? the only reason i use it is because it's been free! now you expect me to pay for them, and how much is that going to cost me?\",\n",
              "  'label': 'card_payment_fee_charged'},\n",
              " {'text': 'i attempted to get money using a foreign currency at an atm but the rate was highly inaccurate!',\n",
              "  'label': 'wrong_exchange_rate_for_cash_withdrawal'},\n",
              " {'text': 'tell me where i can find the auto top up feature and a little bit about it please.',\n",
              "  'label': 'getting_spare_card'}]"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Generating Entire Prompt\n"
      ],
      "metadata": {
        "id": "YrkMoossO01i"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# ! pip install langchain-core\n",
        "# ! pip install openai==0.28"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IwQJwVV_PYy_",
        "outputId": "ac160916-5ac1-4c7f-b9b1-b37e0bee6394"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: openai==0.28 in /usr/local/lib/python3.10/dist-packages (0.28.0)\n",
            "Requirement already satisfied: requests>=2.20 in /usr/local/lib/python3.10/dist-packages (from openai==0.28) (2.31.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from openai==0.28) (4.66.4)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from openai==0.28) (3.9.5)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai==0.28) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai==0.28) (3.7)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai==0.28) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.20->openai==0.28) (2024.6.2)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (1.3.1)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (23.2.0)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (1.4.1)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (6.0.5)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (1.9.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->openai==0.28) (4.0.3)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Helper to format the k-shot prompt with:\n",
        "# - prefix\n",
        "# - 1 example from each class\n",
        "# - target text for classification\n",
        "from langchain_core.prompts import PromptTemplate\n",
        "from langchain_core.prompts.few_shot import FewShotPromptTemplate\n",
        "def get_prompt(examples_pool, text, examples, prefix=\"\"):\n",
        "    prompt_template = PromptTemplate(\n",
        "        input_variables=[\"text\", \"label\"],\n",
        "        template=\"Text: {text}\\nLabel: {label}\",\n",
        "    )\n",
        "\n",
        "    p = FewShotPromptTemplate(\n",
        "        example_prompt = prompt_template,\n",
        "        examples = examples,\n",
        "        prefix = get_prefix(examples_pool, prefix),\n",
        "        suffix = \"Text: {text}\\nLabel:\",\n",
        "        input_variables = ['text'],\n",
        "        )\n",
        "    return p.format(text=text).strip()\n",
        "\n",
        "print(get_prompt(examples_pool, \"Classify this text!\", examples, \"Beware some labels in the examples may be noisy.\"))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PfPIQpoGOg_p",
        "outputId": "aab9be75-7e8e-4274-fde7-5a12995e41d4"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Beware some labels in the examples may be noisy.\n",
            "You can choose the label from: edit_personal_details,declined_transfer,card_payment_fee_charged,wrong_exchange_rate_for_cash_withdrawal,getting_spare_card,cash_withdrawal_charge,verify_top_up,transfer_timing,apple_pay_or_google_pay,card_payment_not_recognised,visa_or_mastercard,reverted_card_payment,transfer_not_received_by_recipient,country_support,wrong_amount_of_cash_received,refund_not_showing_up,card_linking,failed_transfer,exchange_via_app,fiat_currency_support,activate_my_card,direct_debit_payment_not_recognised,balance_not_updated_after_cheque_or_cash_deposit,cash_withdrawal_not_recognised,transfer_fee_charged,card_arrival,pending_top_up,extra_charge_on_statement,supported_cards_and_currencies,declined_card_payment,top_up_failed,automatic_top_up,transaction_charged_twice,disposable_card_limits,card_payment_wrong_exchange_rate,pending_transfer,declined_cash_withdrawal,balance_not_updated_after_bank_transfer,beneficiary_not_allowed,why_verify_identity,order_physical_card,request_refund,card_about_to_expire,cancel_transfer,pending_card_payment,change_pin,lost_or_stolen_phone,pending_cash_withdrawal,top_up_reverted,exchange_charge\n",
            "\n",
            "Text: i just got married and i need to change my name\n",
            "Label: edit_personal_details\n",
            "\n",
            "Text: it declined my transfer.\n",
            "Label: declined_transfer\n",
            "\n",
            "Text: why am i being charged for atm cash withdrawals? the only reason i use it is because it's been free! now you expect me to pay for them, and how much is that going to cost me?\n",
            "Label: card_payment_fee_charged\n",
            "\n",
            "Text: i attempted to get money using a foreign currency at an atm but the rate was highly inaccurate!\n",
            "Label: wrong_exchange_rate_for_cash_withdrawal\n",
            "\n",
            "Text: tell me where i can find the auto top up feature and a little bit about it please.\n",
            "Label: getting_spare_card\n",
            "\n",
            "Text: how come i got charged extra for withdrawing cash?\n",
            "Label: cash_withdrawal_charge\n",
            "\n",
            "Text: i don't know where the top-up verification code is.\n",
            "Label: verify_top_up\n",
            "\n",
            "Text: how long do i have to wait for a us transfer?\n",
            "Label: transfer_timing\n",
            "\n",
            "Text: am i able to do top-ups with google pay?\n",
            "Label: apple_pay_or_google_pay\n",
            "\n",
            "Text: tell me why there is an extra charge when i used an atm?\n",
            "Label: card_payment_not_recognised\n",
            "\n",
            "Text: do you guys accept mastercard or visa?\n",
            "Label: visa_or_mastercard\n",
            "\n",
            "Text: the payment i made with my card is not showing up.\n",
            "Label: reverted_card_payment\n",
            "\n",
            "Text: when i exchanged currencies the value of my money changed.\n",
            "Label: transfer_not_received_by_recipient\n",
            "\n",
            "Text: what countries are you available in?\n",
            "Label: country_support\n",
            "\n",
            "Text: is there a way to change name?\n",
            "Label: wrong_amount_of_cash_received\n",
            "\n",
            "Text: i am still waiting to find out the status of my refund. i have made previous attempts to find out what the status is to no avail. will you please look into this for me?\n",
            "Label: refund_not_showing_up\n",
            "\n",
            "Text: where do i need to go in the app to enter my card info?\n",
            "Label: card_linking\n",
            "\n",
            "Text: what stopped my transfer, from being accepted?\n",
            "Label: failed_transfer\n",
            "\n",
            "Text: if i need gbp instead of aud what do i do?\n",
            "Label: exchange_via_app\n",
            "\n",
            "Text: i want to know how many different currencies my money can be in\n",
            "Label: fiat_currency_support\n",
            "\n",
            "Text: how long will it take to activate my new card?\n",
            "Label: activate_my_card\n",
            "\n",
            "Text: i see a direct debit charge that i did not approve\n",
            "Label: direct_debit_payment_not_recognised\n",
            "\n",
            "Text: i made a cash deposit to my account but i don't see it\n",
            "Label: balance_not_updated_after_cheque_or_cash_deposit\n",
            "\n",
            "Text: there is a withdrawal in my account i didn't make.\n",
            "Label: cash_withdrawal_not_recognised\n",
            "\n",
            "Text: Curabitur dapibus tellus nec sapien condimentum, eget congue metus efficitur.\n",
            "Label: transfer_fee_charged\n",
            "\n",
            "Text: still waiting on my new card\n",
            "Label: card_arrival\n",
            "\n",
            "Text: your top-up function isn't working, it still says pending even though i know my card works.\n",
            "Label: pending_top_up\n",
            "\n",
            "Text: i was charged an extra pound for no reason. why is that?\n",
            "Label: extra_charge_on_statement\n",
            "\n",
            "Text: my card payment doesn't work.\n",
            "Label: supported_cards_and_currencies\n",
            "\n",
            "Text: Did you hear about the mathematician who's afraid of negative numbers? He'll stop at nothing to avoid them!\n",
            "Label: declined_card_payment\n",
            "\n",
            "Text: why is it failing to top-up?\n",
            "Label: top_up_failed\n",
            "\n",
            "Text: i will be traveling.  can i add money automatically?\n",
            "Label: automatic_top_up\n",
            "\n",
            "Text: i got charged in double and want a refund\n",
            "Label: transaction_charged_twice\n",
            "\n",
            "Text: do the disposable cards have any restrictions on them?\n",
            "Label: disposable_card_limits\n",
            "\n",
            "Text: i am not receiving the correct exchange rate on my card.\n",
            "Label: card_payment_wrong_exchange_rate\n",
            "\n",
            "Text: i noticed on my online statement that an atm withdrawal i made early is still pending.  is this accurate?\n",
            "Label: pending_transfer\n",
            "\n",
            "Text: the cash i tried to draw from the atm was not approved.\n",
            "Label: declined_cash_withdrawal\n",
            "\n",
            "Text: my balance didn't change after i transferred some money.\n",
            "Label: balance_not_updated_after_bank_transfer\n",
            "\n",
            "Text: the transfer to my beneficiary has not gone through\n",
            "Label: beneficiary_not_allowed\n",
            "\n",
            "Text: am i supposed to verify my identity?\n",
            "Label: why_verify_identity\n",
            "\n",
            "Text: Unaitwa nani?\n",
            "Label: order_physical_card\n",
            "\n",
            "Text: please could you give me a refund\n",
            "Label: request_refund\n",
            "\n",
            "Text: my transfer got declined.\n",
            "Label: card_about_to_expire\n",
            "\n",
            "Text: Nullam vitae turpis sit amet diam luctus efficitur id id lacus.\n",
            "Label: cancel_transfer\n",
            "\n",
            "Text: hello, i made a payment with my card and it's not working. it's been a while and still says its pending. can you tell when this will go through?\n",
            "Label: pending_card_payment\n",
            "\n",
            "Text: can i change my pin through the app?\n",
            "Label: change_pin\n",
            "\n",
            "Text: lost phone, can i still access account?\n",
            "Label: lost_or_stolen_phone\n",
            "\n",
            "Text: i didn't receive my money earlier and it says the transaction is still in progress. can you fix it?\n",
            "Label: pending_cash_withdrawal\n",
            "\n",
            "Text: there is an urgent thing that i need to transfer from china. approximately how long will it take?\n",
            "Label: top_up_reverted\n",
            "\n",
            "Text: is there a pay structure for currency exchanges?\n",
            "Label: exchange_charge\n",
            "\n",
            "Text: Classify this text!\n",
            "Label:\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Query OpenAI API\n"
      ],
      "metadata": {
        "id": "um2RH95dQHXL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import openai, os\n",
        "import string\n",
        "# Helper method to prompt OpenAI LLM and get responses.\n",
        "def get_response(prompt):\n",
        "    response = openai.Completion.create(\n",
        "      model=\"text-davinci-003\",\n",
        "      prompt=prompt,\n",
        "      temperature=0,\n",
        "      max_tokens=50,\n",
        "      top_p=1,\n",
        "      frequency_penalty=0,\n",
        "      presence_penalty=0\n",
        "    )\n",
        "\n",
        "    # Parse output to get just the label.\n",
        "    resp = response['choices'][0]['text'].split('\\n')[0].split(',')[0].strip().lower().rstrip(string.punctuation)\n",
        "\n",
        "    # Just in case a respone is not a perfect match, we know.\n",
        "    if resp not in examples_pool.label.unique():\n",
        "        print(resp)\n",
        "    return resp\n",
        "\n",
        "text = \"\\'How can I change my pin?\\'\"\n",
        "examples = get_examples(examples_pool)\n",
        "prompt = get_prompt(examples_pool, text, examples)\n",
        "response = get_response(prompt)\n",
        "print(\"Model classified \", text, \" as \", response)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 443
        },
        "id": "GRxa9uEpP5ix",
        "outputId": "4c975b16-0699-42ef-bdae-368fbe609643"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "error",
          "ename": "AuthenticationError",
          "evalue": "No API key provided. You can set your API key in code using 'openai.api_key = <API-KEY>', or you can set the environment variable OPENAI_API_KEY=<API-KEY>). If your API key is stored in a file, you can point the openai module at it with 'openai.api_key_path = <PATH>'. You can generate API keys in the OpenAI web interface. See https://platform.openai.com/account/api-keys for details.",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAuthenticationError\u001b[0m                       Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-8-27ca53ca3dc3>\u001b[0m in \u001b[0;36m<cell line: 26>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     24\u001b[0m \u001b[0mexamples\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_examples\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexamples_pool\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     25\u001b[0m \u001b[0mprompt\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_prompt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexamples_pool\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtext\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexamples\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 26\u001b[0;31m \u001b[0mresponse\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_response\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprompt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     27\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Model classified \"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtext\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\" as \"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mresponse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m<ipython-input-8-27ca53ca3dc3>\u001b[0m in \u001b[0;36mget_response\u001b[0;34m(prompt)\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;31m# Helper method to prompt OpenAI LLM and get responses.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mget_response\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprompt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m     response = openai.Completion.create(\n\u001b[0m\u001b[1;32m      6\u001b[0m       \u001b[0mmodel\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"text-davinci-003\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m       \u001b[0mprompt\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mprompt\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/openai/api_resources/completion.py\u001b[0m in \u001b[0;36mcreate\u001b[0;34m(cls, *args, **kwargs)\u001b[0m\n\u001b[1;32m     23\u001b[0m         \u001b[0;32mwhile\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     24\u001b[0m             \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 25\u001b[0;31m                 \u001b[0;32mreturn\u001b[0m \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcreate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     26\u001b[0m             \u001b[0;32mexcept\u001b[0m \u001b[0mTryAgain\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     27\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mtimeout\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0mstart\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mtimeout\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/openai/api_resources/abstract/engine_api_resource.py\u001b[0m in \u001b[0;36mcreate\u001b[0;34m(cls, api_key, api_base, api_type, request_id, api_version, organization, **params)\u001b[0m\n\u001b[1;32m    147\u001b[0m             \u001b[0murl\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    148\u001b[0m             \u001b[0mparams\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 149\u001b[0;31m         \u001b[0;34m)\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcls\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__prepare_create_request\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    150\u001b[0m             \u001b[0mapi_key\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mapi_base\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mapi_type\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mapi_version\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morganization\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mparams\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    151\u001b[0m         )\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/openai/api_resources/abstract/engine_api_resource.py\u001b[0m in \u001b[0;36m__prepare_create_request\u001b[0;34m(cls, api_key, api_base, api_type, api_version, organization, **params)\u001b[0m\n\u001b[1;32m    104\u001b[0m             \u001b[0mparams\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"timeout\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mMAX_TIMEOUT\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    105\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 106\u001b[0;31m         requestor = api_requestor.APIRequestor(\n\u001b[0m\u001b[1;32m    107\u001b[0m             \u001b[0mapi_key\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    108\u001b[0m             \u001b[0mapi_base\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mapi_base\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/openai/api_requestor.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, key, api_base, api_type, api_version, organization)\u001b[0m\n\u001b[1;32m    136\u001b[0m     ):\n\u001b[1;32m    137\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapi_base\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mapi_base\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mopenai\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapi_base\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 138\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapi_key\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkey\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mutil\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdefault_api_key\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    139\u001b[0m         self.api_type = (\n\u001b[1;32m    140\u001b[0m             \u001b[0mApiType\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_str\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mapi_type\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/openai/util.py\u001b[0m in \u001b[0;36mdefault_api_key\u001b[0;34m()\u001b[0m\n\u001b[1;32m    184\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mopenai\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mapi_key\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    185\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 186\u001b[0;31m         raise openai.error.AuthenticationError(\n\u001b[0m\u001b[1;32m    187\u001b[0m             \u001b[0;34m\"No API key provided. You can set your API key in code using 'openai.api_key = <API-KEY>', or you can set the environment variable OPENAI_API_KEY=<API-KEY>). If your API key is stored in a file, you can point the openai module at it with 'openai.api_key_path = <PATH>'. You can generate API keys in the OpenAI web interface. See https://platform.openai.com/account/api-keys for details.\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    188\u001b[0m         )\n",
            "\u001b[0;31mAuthenticationError\u001b[0m: No API key provided. You can set your API key in code using 'openai.api_key = <API-KEY>', or you can set the environment variable OPENAI_API_KEY=<API-KEY>). If your API key is stored in a file, you can point the openai module at it with 'openai.api_key_path = <PATH>'. You can generate API keys in the OpenAI web interface. See https://platform.openai.com/account/api-keys for details."
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "u40mCwAZQZT5"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}